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^ ■ Abstract 



An adaptive nonparametric estimation procedure is constructed 
for heteroscedastic regression when the noise variance depends on the 
unknown regression. A non-asymptotic upper bound for a quadratic 
risk (oracle inequahty) is obtained. 
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1 Introduction 

Suppose we are given observations (%)i<j<„ which obey the heteroscedastic 
regression equation 

Vi = S{xj) + aj{S)ij , (1.1) 

where design points Xj = j/n, S{-) is an unknown function to be estimated, 
(^j)i<7<n is ^ sequence of i.i.d. random variables, (crj(5'))^< •<„ are unknown 
volatility coefficients depending on unknown regression function S. 

The models of type (11. ip with <7j{S) = crj{xj) were introduced in Akri- 
tas. Van Keilegom (2001) as a generalisation of the nonparametric ANCOVA 
model of Young and Bowman (1995). It should be noted that heteroscedastic 
regressions with this type of volatility coefficients have been encountered in 
econometric studies, namely, in consumer budget studies utilizing observa- 
tions on individuals with diverse incomes and in analyses of the investment 
behavior of firms of different sizes (see Goldfeld, Quandt, 1972). For example, 
for consumer budget problems one uses there (see p. 83) some parametric 
version of model (II. ip with the volatility coefficient defined as 

cr^iS) = c, + c,x^+c,S\x^), (1.2) 

where Cq, c^ and Cg are some positive unknown constants. 

Moreover, this regression model appears in the drift estimation problem 
for stochastic differential equations when one passes from continuous time 
to discrete time model by making use of sequential kernel estimators having 
asymptotically minimal variances (see Galtchouk, Pergamenshchikov, 2004; 
2006; 2007a; 2007b). 

The volatility coefficient estimation in heteroscedastic regression was con- 
sidered in a few papers (see, for example, Cai,Wang, 2008 and the references 
therein). By making use of the squared first-order differences of the observa- 
tions the initial problem in that paper was reduced to the regression function 
estimation in the model of type (II. 11) . 

In this paper we develop the approach proposed in Galtchouk, Perga- 
menshchikov (2005). The first goal of the research is to construct an adap- 
tive procedure based on observations (|/j)i<j<„ for estimating the function 
S and to obtain a sharp non-asymptotic upper bound (oracle inequahty) for 
a quadratic risk in the case when the smoothness of S is unknown. The 
second goal is to prove that the constructed procedure is efficient also in the 
asymptotic setup. 



Problems of constructing a nonparametric estimator and proving a non- 
asymptotic upper bound for a risk in homoscedastic model, that is when 
aj{S) = a, were studied in few papers. A non-asymptotic upper bound 
for a quadratic risk over thresholding estimators is given in Kalifa, Mallat 
(2003). In papers by Barron, Birge, Massart (1999), Massart (2004) an 
adaptive model selection procedure has been constructed. It is based on least 
squares estimators and a non-asymptotic upper bound has been obtained for 
a quadratic risk which is best in the principal term for the given class of 
estimators when the noise vector (^i . . . , ^„) is gaussian. This type of upper 
bounds is called the oracle inequality. In Fourdrinier, Pergamenshchikov 
(2007) the oracle inequality has been obtained for a model selection procedure 
based on any estimators in the case when the noise vector (^i, . . . , ^„) has a 
spherically symmetric distribution. Moreover, some sharp oracle inequalities 
have been obtained also for homoscedastic regression with gaussian noises, 
see, for example, Kneip (1994). Here the adjective "sharp" means that the 
coefficient of the principal term may be chosen as close to unity as desired. 

In the paper for heteroscedastic regression an adaptive procedure is con- 
structed for which the sharp non-asymptotic oracle inequality is proved. It 
should be noted that the methods used in former papers to obtain the sharp 
oracle inequality in regression models are limited by the homoscedastic case 
since they are based on the fact that an orthogonal transformation of a noise 
gaussian vector (^i, . . . , ,^„) gives a gaussian vector. In heteroscedastic regres- 
sion models under consideration these methods are not valid since the noise 
vector is not gaussian. To obtain sharp non-asymptotic oracle inequalities in 
the heteroscedastic case the authors develop a new mathematical tools based 
on "penalty" methods and Pinsker's type weights. 

Moreover, in Galtchouk, Pergamenshchikov (2007c) we show that the 
given adaptive estimator is efficient in the asymptotic sense, that is, the sharp 
asymptotic lower bound is proved for a quadratic risk and it is attained over 
this estimator. 

The paper is organized as follows. In Section 2 we construct an adaptive 
estimation procedure based on weighted least squares estimators and we ob- 
tain a non-asymptotic upper bound for the quadratic risk. In Section 3 we 
propose an estimator for the summarized noise variance and give the oracle 
inequality in the case of Sobolev space, S G W''. The proofs are given in 
Section 4. The Appendix contains some technical results. 



2 Oracle inequality 

In this paper we study the non-asymptotic estimation problem of the function 
S in the model (11.11) by observations {yj)i<_j<_n with odd sample number n. 
We assume that in (11. ip the sequence (Cj)i<j<„ is i.i.d. with 

E^i = , E^l = l and E^f = ^* < oo . (2.1) 



In the sequel we denote by ^ = V^* — 1. 

Moreover, we assume that (o"i(S'))]^<;<„ is a sequence of positive random 
variables independent of (6)i<j<n ^^^ bounded away from +oo, i.e. there 
exists some nonrandom unknown constant 0"=,, > 1 such that 

max af(S) < a^ . (2.2) 

l<l<n '■ 

For any estimate 5'„ of S based on observations (yj)i<j<„, the estimation 
accuracy is measured by the mean integrated squared error (MISE) 

Es\\Sn-S\\l, (2.3) 

where 

^ ^ 1 " - 

ll-^^ - '^ll^ = (^„ -S,S^- S)n = - y2iS„{xi) - S{xi)f . 

n ^ — ^ 
1=1 

We make use of the trigonometric basis (0j)~>i in C2[0, 1] with 

01 = 1 , 0^.(x) = V2Tr^{2n[j/2]x) , j > 2 , (2.4) 

where the function TrAx) = cos(x) for even j and TrAx) = sin(a;) for odd 
j] [x] denotes the integer part of x. Note that if n is odd, then this basis is 
orthonormal for the empirical inner product generated by the sieve {xX^-^^, 
that is for any 1 <i,j < n, 



(0., 0.)n = -X^0.(a;O0,(^O = Kr,, , (2.5) 

n ^ — ' 
1=1 

where Kr- is Kronecker's symbol. 



By making use of this basis we define the discrete Fourier transformation 
in fll.ll) and obtain the Fourier coefficients 

Oj,n = iy,<l^j)n and ^,.„=(^,0,)n. (2.6) 

Here Y = (y^, . . . , y^)' and S = {S{x^), . . . , S'(x„))'. The prime denotes the 
transposition. 

/^From fll.ll) it follows directly that these Fourier coefficients satisfy the 
following equation 

^j,n = dj,n + -^^3,n (2-7) 

with 



e 



1 
^"^ 1=1 



We estimate the function S by the weighted least squares estimator 

n 

S,{x) = Y.X{j)e^^J^{x), (2.8) 

i=i 

where x G [0,1], the weight vector A = (A(l), . . . , A(?2))' belongs to some 
finite set A from [0, 1]". We denote by u the cardinal number of the set A. 
Moreover, we set 

n n 

g^ = max Y] A(j) and ^ = max sup | V" A*(j)0(x)| , (2.9) 

j=i j=i 

where (^^ =0^ — 1 and i = 1,2. 

Now we need to write a cost function to choose a weight A G A. Of course, 
it is obvious, that the best way is to minimize the cost function which is equal 
to the empirical squared error 



Err„(A) = ||5,-5 
which in our case is equal to 



Err.(A) = J^ ^'0')^,n " 2 J^ Mj)^„n 0,,n + E C • (2.10) 

j=i j=i j=i 



Since coefficients 6- „ are unknown, we need to replace the term 6- „ 9- „ by 
some estimator which we choose as 



5^ 1 



^j,n = ^i.n - -^n 



where ^ is some estimator of the summarized noise variance 

n 

<^n = n-' Y. ^f(^y (2.11) 

1=1 

Such type of estimators is given in (13.51) . 

Moreover, for this substitution to the empirical squared error one needs 
to pay a penalty. Finally, we define the cost function by the following way 

n n 

JnW = E ^'(^■)^,n - 2 E ^(^■) ^.." + P^nW , (2-12) 

i=i i=i 

where p is some positive coefficient which will be chosen later. The penalty 
term we define as 

I \ 1 2^ " 

P„(A) = Ui^ with |Ap = EA^O-)- (2.13) 

i=i 

Note that in the case when the sequence ((7/(>S'))]^<^<„ is known, i.e. ^ = <;"„, 
we obtain 

PJA) = ^^. (2.14) 

n 

We set ^ 

A = argmin^g^ J„(A) (2.15) 

and define an estimator of S as 

^. = ^5;. (2.16) 

We recall that the set A is finite so A exists. In the case when A is not unique 
we take one of them. 

To formulate the oracle inequality we introduce, for < p < 1/3, the 
following function 

KiP) = ^^^:r^^ -. (2.17) 
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with 



T»-^ + 4,,,„(l + .-|)+4.,„-| 



Theorem 2.1. Let A be any finite set in [0, 1]". For any n > 3 and 

< p < 1/3, the estimator S^ satisfies the oracle inequality 

Es\\S^ -S\\1<{1 + K{p)) minE^II^, - S\\l + l B^{p) , (2.18) 

where B^{p) = ^„(p) + kMQu ^s\^n-'^n\ with 

<p) = 'i^ and .,(p)=4i^. 

If in model f 1 1.1 1) the volatility coefficients (o"j(5'))]^<;<„ are known, then 
^n — '^n and inequality fl2.18p has the following form 

Es\\S. -S\\1<{1 + «:(p)) minE^||5, - S\\l + ^vl/jp) . (2.19) 

Remark 2.1. Note that the principal term in the right-hand side of (I2.18P - 
(I2.19P is best in the class of estimators {S;^^ , A e A). Inequalities of such type 
are called the sharp non-asymptotic oracle inequahties. The inequality is 
sharp in the sense that the coefficient of the principal term may be chosen as 
close to 1 as desired. Similar inequalities for homoscedastic models (II. ip with 
(Ji{S) = o were given, for example, in [T^ . The methods used there cannot 
be extended to the heteroscedastic case since, after the Fourier transforma- 
tion, the random variables (^^ „) in model (12. 7p are dependent contrary to the 
homoscedastic case, where these random variables are independent (see, for 
example, Rohde, 2004)- 

Remark 2.2. // one would like to obtain the asymptotically minimal MISE 
of the estimator S^, then the secondary term B^{p) in (I2.18P should be slowly 
varing. Indeed, since usually the optimal rate is of order 77,2*-7(2fc+i) j^^ some 
k > 1, then after multiplying the inequality (I2.18P by this rate the principal 
term gives the optimal constant and the secondary one should be of type that 
for any 6 > 

J > as n ^ oo . 



Due to the definitions ^I/„(p) and B,^{p), it should he, for any 5 > Q, 

n° 
One can take, for example, the parameter p tending to zero as n —>• oo like 



1 



P=0{,-Tz] (2-20) 



for some 7 > 0. The choice of g^ and of the estimator ^ is proposed below. 

Consider now the order of the termes QnyQin''^2n ^^^ ^^^ function \I/„(p) 
in the case when the finite set A is formed by a special version of Pinsker's 
weights (see, for example, [I5]). To this end, we define the sieve 

Ji^ = |i, . . . , K^f X |ix, . . . , traj , 

where ti = ie and m = [l/e^]. We suppose that the parameters k^ > 1 and 
< £ < 1 are functions of n such that, 

lim„^^ k^ = +oo, lim„_^ ife = , 

(2.21) 
lim„^^ e = and lim„^^ n^e = +00 , 



for any 6 > 0. For example, one can take e = 1/lnn and k^ = y/lnn for 
n>3. 

For any a = (/5, t) G A^ we define the weight vector A^ = (Aq,(1), . . . , A^(n))' 
as 

KU) = i{i<,<,,} + (1 - UM') lo,<,<..} , (2.22) 

where jo = jo{a) = [c^^/lnn]. 

Hence, 

A = {A„,«e^J (2.23) 

and u = k^m. Note that in this case in view of (I2.2ip for any 5 > 

lim — T = . 



Moreover, by (1^^ 

Therefore, taking into account that An < Ai < 1 for /? > 1 we find that 
i.e. for any 5 > 

n— >oo /i ' 

Moreover, note that for any x G [0, 1], we get 

n Jo 

+i{^.>i} E (i-(jKn0,(^)- 

Thus Lemma [A. 21 iniphes that 

-g, < 1 + 2^+'^ < 1 + 2'=-+^ . 

'-l,n — — 

Due to the condition for k^ in fl2.2ip this function is slowly varying, i.e. for 
any 6 > 0, 



hm % = 0, 

n— »oo Tl 



By the same way we obtain that 



g. 



2,n 



< 1 + 2'=*+2 ^ 22^=*+^ 



and, therefore, for any 6 > 



lim — ^ = - 

n— »oo ?^ 



Thus, if we choose the parameter p = pn as in fl2.20p we obtain that in this 

case, for any S > 0, 

^ (p) 
lim li^ = o. (2.24) 

n— >oo n 



3 Oracle inequality for S G W^ 

Assume that S : M — > M is a A; times differentiable 1-periodic function such 
that 

k 

5^||5(^-)f <r, (3.1) 

i=o 
where 

fmt. (3.2) 

'0 

We denote by W^^ the set of all such functions. Moreover, we suppose that 
r > and k > 1 are unknown parameters. 

Note that, the space W^ can be represented as an ellipses in the Hilbert 
space, i.e. 

W^ = {S eC^lO,!] : S = J2 ^j<Pj such that ^ a/^ < ^| ^ (33) 

i=i i=i 

where the basis functions {4>j)j>i are defined in (12.41) : {Oj)j>i are the Fourier 
coefficients, i.e. 

e^ = {SAj)= f 5(%(t)dt. (3.4) 

The coefficients (aj)j>i are defined as 

k k 

a 



1=0 1=0 

To estimate i;^, we make use of the following estimator: 



?n= E ^,.' (3-5) 

i=m„+l 



where the parameter 1 < m^ < n will be chosen later. 
In Section m we show the following result. 

Lemma 3.1. For any n > 2 and r > 0, 



, a + q*(r) ^ ^ 

sup Es\^^-^J< ^, (3.6) 
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where a = 2 (^ + -\/2) a^ and 

If we choose the parameter m„ in (13.51) such that 

m m? 

hm — ^ = and hm — ^ = oo , (3.7) 

n— >oo \/Ti n— >oo \/TI 



we obtain that 

hm ^*(r) = . 

n—foc 

Theorem 12.11 and inequahty (13.61) imply immediately the following result. 

Theorem 3.2. Let A be any finite set in [0, 1]". Assume that in the model 
(II. ip the function S belongs to W\ Then, for any n > 3 and < p < 1/3, 

the procedure S^ from (12.161) with ^ defined by (13.51) and (13.71) satisfies the 
following oracle inequality 

Es\\S, - S\\l < (1 + Kip)) minE^II^, - S\\l + -V^{p,r) , (3.8) 

agA n 

where 

// the set A is from (12.231) . then for any 6 > and any < p < 1/3 



4 Proofs 

4.1 Proof of Theorem 12.11 

First of all, note that we can represent the empirical squared error Err„(A) 
by the following way 

n 

Err„(A) = J„(A) + 2 J] A(j)e;.„ + 1|51|^ - pP„(A) (4.1) 

i=i 

11 



^th ^;. „ = ^,,„ - ^,. J,,„. By setting 

S-,n = E5C = ^i:-?(^)0?(^O, (4.2) 

we find that 



n 
=1 



where ^ ^ 

^i,n = C~S-.n a^d ^i,n = S-,n - ^n • (4-3) 

Note now that, we can represent ^^-^ as 

In = - E ^?(5)</>?(^0el + 2 5: r^,,e. = §,„ + 2^;; , (4.4) 



n 

1=1 1=2 



where C,i = ^f — I and 



1 '"' 



n 

d=l 



Now we set 

iVi(A) = $:^0')C ^^d ^2(A) = -=E^(^')C1k>o}. (4.5) 

where A(j) = A(j)/|A|. In the Appendix we show that 

supE5|iVi(A)| <e^,(f?„+^^J^ (4.6) 

aga 



n 



and 

sup E5(A/'2(A))^ <2(T*/n. (4.7) 



Now, for any A G A, we rewrite (14. ip as 



Err,(A) = J„(A) + ^Ar,(A) + Ay^Pjr)N,i\) 
+ 2M{X) + ^AiX) + \\S\\l-pPM, 



12 



where -P„(A) is defined in (12.141) . 

n n 

A{\) = Y,mhn and M(A) = n-^/2 5^A(j)^,.„e„„. (4.8) 

We start with A (A). Setting 

1 " - 

we obtain that 

n 
|A(A)| < I Y^ A(j)S> I + 0n\^n - "^nl 

<o^*^i,„ + &„!?„-?„ I- (4.10) 

Now from (14. ip we obtain that, for some fixed Aq € A, 

ErrJA) - Err JAq) = J(A) - J(Ao) + 2M(^) + ^N,{d) 

- pP„(A) + pP„(Ao) + I (a(A) - A(Ao) 

where 'd = \ — Xq. 

By the definition of A in (12.151) and by (I4.10p we get 

ErrJA) - Err„(Ao) < 2M(^) + ^" 



2 



+ -iV,(^)+4VP„(A)iV2(A)-pP„(A) 



+ pP„(Ao)-4v/P„(Ao)iV2(Ao). 
Moreover, making use of the inequahty 

2\ab\ < ea^ + e'^b^ (4.11) 



13 



with e = p/4 and taking into account the definition of penalty term in (12.131) 
we deduce, for any A G A, 

4-/PJXJ\N,{X)\ < pPJX) + 4^ 

Thus from here it follows that 

Err„(A) < ErrJAo) + 2M{d) + T„ + 2pP„(Ao) , (4.12) 

where _ 

n p n n 

with A^* = sup^g^ l^i(-^)l and N* = sup^^^ |A^2('^)I- Moreover, note that the 
bounds dm), (gil) and flTTOj) imply that 

E^T^ < T;(p)^ + ^^„E,|?„ - ,J , (4.13) 

where the function T*(p) is defined in (12.171) . 

Now we study the second term in (14. 8p . First, note that for any nonran- 
dom vector i) = {^{1), ■ ■ ■ , ^(n))' G MJ^ Lemma [A. 41 implies 

rr "■ IK l|2 

EsM\^) < ^ J2^\j)9l = aj^ , (4.14) 



n '■ — ' ■" n 



where 



We set now 






II C 112 
i?eAi ll^)?lln 

We estimate this term with the help of inequality (14.141) . i.e. 



sup with A^ = A — Aq 

0,q 






^€A, " '^"" 
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Moreover, making use of inequality (14.111) with e = p||5'^||„, we get 

2|M(^)|<p||5,||^ + — . (4.16) 

" np 

Now we estimate IIS*,,, IP. We have 

II -Clin 

n 

11^.11' - 11^.11' = E ^'(3){0l - Pj < -2M,(^) (4.17) 



with 



1 " 



Now, taking into account that \'&{j)\ < 1 for any {} G A^, we obtain 

^ n 

Putting 



Zl = sup 



we get 

EgZ* < ua,. (4.18) 

Therefore, applying inequality 04.161) for M^(t9) in (14.171) we deduce the upper 
bound for IIS*,, IP, i.e. 

II wlin' 

IISJP < -^\\SJ^ + -^ — -. (4.19) 

II ^lln - l_p\\ <?lln np(l-p) ^ ^ 

Taking into account this inequality in (I4.16P we obtain that 



2M(^)< T^\\SJl + 



z* + z: 



1- p '^ " np(l -p) 
^2p(Err„(A) + Err„(Ao)) , Z* + Z; 



1 - p np{l - p) 



15 



Therefore (I4.12p implies that 



z- + z; ^ 2p(i-p ) 

np{l — 3p) 1 — 3p 



+ r^7T^^ + ^^;^^^n.(\). 



Now by inequahties fl4.15p - fl4.18l) we get that 

E^ErrJA) < i^E^ErrJAo) + f^^^^ "^"^ 
+ ^^^^^ + Mi^EPfAl 

By making use of inequahty fl4.13p and Lemma lA.ll we come to Theorem 12.11 
D 



4.2 Proof of Lemma 13.11 

First notice that from (12. 7p we obtain that 

n Q '^ 

^n~ '^n — 2-^ j,n ~^ 77n ^-^ ■''" -^'^ 



r . — — Ilr 



i=m„+l i=ni„+l 

ra n 

+ ^"' E 5,n + ^"' E 

i=m„+l i=m„+l 

2 ^ 1 1 m„ 

:= Ai + ^A2 + -A3 + -A4 ^ ?„ , 



where ^j- „ and ^^ „ are defined in (14. 3 p and (14.90 respectively. 

We estimate the first term by Lemma [A. 31 for S G W^. We have 

4r 
Ai<— . 

n 

The next term we estimate with the help of Lemma IA.4I We get that 

Ar 
EsiA,f<(r,A,<a,— . 

n 

16 



By ( I4.4p and (14.51) we can represent A3 as 

A, = N,{\,) + 2\\,\,/^N,{\j) 

with the vector \j = (Aj(l) , . . . , Aj(n))' having the indicator components, 
i.e. Xj{j) = l{j>m }• By estimating inl JA.ip 0^ by 2 we obtain 

Thus the upper bound (14. 7p imphes 

E5IA3I < 2(x^{^ + V2)y/n = Wy/n . 
Moreover, due to Lemma [A. 21 with m = 0, one has 



lA. 



n 



-'E-Ks) E 



[Xdj 



d=l 



j=m„+l 






n 



d=i 






+ -E 



d=l 



Yl M^'^^ 



<2(T*. 



Hence Lemma [3. II D 



5 Appendix 



A.l Proof of (USD 

First note that we can represent the term N^{X) as 



^i(^) = Yl '"i^r.ii with Vi^^^ 



af{S) 



1=1 



n 



E^(^>?( 



i=i 



-2 



Recalhng that E,^^ = ^* — 1 = ^' we calculate 



1=1 



.i=i 
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Therefore for any vector A G M"" 

= max I 

1 0<a:<l 



Es\N,{X)\<a,^ max iJ^KMi^)]- (A.l) 

^yn 0<a:<l ^-^ ^ 



Thus taking into account here definitions (12.91) we come to inequahty (14. 6p . 
D 

A.2 Proof of I K7\ ) 

By putting a^ = J2"^-i '^U)'^j,i ^"^^ taking into account that the random 
variables {ik)i<k<n ^^^ independent of (o"fc('S'))i<A:<n "^^ obtain that 

(n \ ""^ " 

5^af(5) 5^S,, (A.2) 
/=i / j=i 

where 

«; = E(af I (T;t(^) , 1 < A; < n) 

Therefore the orthonormahty property (12.51) imphes that for any A G R" 



af(5) 



n I n 



it I 

d=l \j=l 

n ^ — ^ •' n 

Now by making use of this inequahty in flA.2p we get (14.71) . 
D 



A. 3 Technical lemma 
Lemma A.l. For any n > 1 and X E A, 

At f o 

Proof. Indeed, by the definition of Err„(A) we have 

Err„(A) = ^ ( (A(j) - 1)^^. „ + A(j)^e, J • 
Therefore, 

E,Err„(A) > E,- J^ ^'U) C = E,- J2 ^'0) S'. 



i=i 



i=i 



where the sequence {^j^n) i^ defined in (I4.2p . Moreover, note that the last 
term can be estimated as 






_. n n 



< cr^g. 



2,ra 



We recall that the definition of the set A and the definition of p„ in (12.91) 
imply that |Ap < g^ for A G A. Therefore for any A G A 

n 

Yl A^(j) .jj- „ > \X\% - a^-g^^^ - |An?„ - ^„| 

> |Ar?„ - 0-*^2,n - ^nl?n " ^nl • 

Hence the desired inequality. D 

A. 4 Properties of trigonometric basis 
Lemma A. 2. For any m > 0, 



sup sup A^ 

N>2 xG[0,l] 



N 



E'' 

1=2 



[X 



< 2™. 



(A.3) 
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Proof. Due to the properties of the trigonometric functions, we get 



1=2 



= J2 (2/)" cos(47r/x) 

l<l<N/2 

J2 (2/ + l)™cos(47r/a;) 

l<l<{N-l)/2 



This yields 

N 



X] 



1=2 



< E ((2/ + l)'^-(2/)'^)cos(47r/x) 

l<K(Af-l)/2 

< E ((2/ + 1)™ - (2/)™) + AT'" 

l<K{Ar-l)/2 
l<«<(Af-l)/2 i=0 ^ ^ 



+ N" 



This imphes ( 1A31) . 
D 



Lemma A. 3. For any function S G W^ , 



sup sup m 

n>l l<m<n— 1 



2fe 



y 9' 



< 



Ar 



\j=m+l 



^2(k~l) 



(A.4) 



Proof. First, note that any function S from W^!' can be represented by its 
Fourier series, i.e. S = YlT=i^3't'j with the coefficients defined by (13.41) . By 
denoting the residual term for S as 



A„(x) = S - ^Oj(f)j = E ^j(t^ji^) 

j=l j=m+l 



we obtain that 



E ^■„= inf ||5-E«.>.lln < l|Arn||^ 



j=m+l 



i=i 
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Moreover, it is easy to deduce that 



n n 



k=l k=l "'^fe-i 

< 2 / A^(x)dx + 2^ / (A^ixk) - A^{x)Ydx . 

"^0 fc=l "^^fc-i 

The last term in this inequahty we estimate as 

(A„(xfc) - A„(x))2 = ( / ' ^-(-2)^2; 



Therefore, 



2 

IIA IP < 2IIA IP -I HA 11^ 

j=m+l j=m+l 

Now note that by the representation of the set W^ in the form fl3.3p we can 
estimate the first term in the last inequality as 

OO OO 

Similarly, we find that 

y 9^^^\'< sup ^r< sup ^ir<^ ^^^-^ . 

Therefore, for m < n we get that 

^ OO 

This imphes fETij) . D 
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OO 

2|| J, l|2 ^ ''" 






Lemma A. 4. Let ^,- he defined in f l2.7p for the model fll.ll) . Then, for any 

real numbers f i, . . . , w„, 

/ n \ 2 n 

Proof. Due to the definition of ^^ „, one lias 









E ^i ^hn = E ^'^' ^' 








j=l i=l 


with 






1 " 

^' - /^ E MA^i) ■ 


Moreover 












/ n 


\ 2 n n 




E 


[^ 


.,e,v. = y^ af (^)^f < a. 5^ 






Vi=i 


/ i=l Z=l 

n 



The orthogonality of the basis {(pj) implies inequality (lA.Sp . Hence Leninia rA.4[ 
D 
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